Image processing system, method of controlling the image processing system, and program for a peripheral apparatus in the system

ABSTRACT

The image processing apparatus  1  at the manufacturing side can be accessed over the Internet  3  by the peripheral apparatuses  2 A,  2 B, and  2 C at remote locations. When the image processing apparatus  1  receives from any of the peripheral apparatuses  2 A,  2 B, and  2 C an image transmission command based on the HTTP protocol, it generates and transmits a response to the command carrying an image to the commanding peripheral apparatus  2 A,  2 B, or  2 C. The peripheral apparatus  2 A,  2 B, or  2 C upon being energized repeats an action of dispatching another image transmission command, an action of receiving the response, and an action of displaying the display image carried in the response in this order.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a technology for processing with theuse of a computer an image produced through shooting an object ofinterest to measure and analyze the object. Particularly, the presentinvention relates to a technology for controlling the communications andthe processing action between an image processing apparatus and adesired number of peripheral apparatuses in a system which are connectedtogether over a computer network for conducting the measurement action.

2. Description of the Related Art

An image processing apparatus is known as a “visual sensor” and commonlyused for positioning assembling components or inspecting finishedproducts at the manufacturing site in the industrial field. Such animage processing apparatus is designed for subjecting an image ofinterest produced by a camera to binary processing and edge extractingto determine the characteristics and then measuring the location andsize of an object from the characteristics. Also, when used forinspection or test, the image processing apparatus can judge fromcomparing between the measurements and reference thresholds whether theobject is good or not before releasing the result of judgment.

However, such a conventional image processing apparatus is equipped witha limited interface such as RS-232C format and can be communicated withexternal apparatuses only within a given area. For compensation, theapplicants of the present invention have developed a modified imageprocessing apparatus provided with an interface connectable with theEthernet (a registered trade name) (See “Non-stop adjusting function ofvisual sensor, model F250” by Go Yoshiura, Yusuke Iida, and YukihiroKomatsu, Omron Technics, OMRON Corporation Technology HQ, Vol. 41, No.3, September 2001, pp. 266-270). The modified image processing apparatusis installed in a LAN system at the manufacturing site and can thus beincreased in the area of communications for exchanging information.

The modified image processing apparatus disclosed in “Non-stop adjustingfunction of visual sensor, model F250” by Go Yoshiura, Yusuke Iida, andYukihiro Komatsu, Omron Technics, OMRON Corporation Technology HQ, Vol.41, No. 3, September 2001, pp. 266-270 is connectable over the Internetto a peripheral apparatus at a remote location (distanced far from themanufacturing site where the image processing apparatus is located).When the image processing apparatus is connected to a remote location,it can be operated by an expert who is absent at the manufacturing sitefor monitoring an image of interest and modifying the setting foroperations, hence improving the expediency.

The modified processing apparatus may yet fail to connect with aperipheral or computer system of a user at the remote location which isprotected with a means, such as a firewall or a router, for limiting theinput of external data. In addition, the peripheral apparatus used bythe user at the remote location is mostly a personal computer and canscarcely allow a worker at the manufacturing site to monitor the actionof the user entering or modifying the setting data.

SUMMARY OF THE INVENTION

The present invention has been developed in view of the above aspectsand its primary object is to ensure the communications between an imageprocessing apparatus and peripheral apparatus(es) regardless of thenetwork environments of each user while identical images of interest canbe viewed at substantially the same time at both the location of theimage processing apparatus and the remote location of each peripheralapparatus.

It is a secondary object of the present invention to allow the displayscreens of both an image processing apparatus and each peripheralapparatus to be updated according the setting action which is carriedout by a user at a remote location so that another user at themanufacturing site can acknowledge the setting action.

An image processing system according to the present invention comprisesan image processing apparatus and a desired number of peripheralapparatuses connected together over a computer network for carrying outa given measurement action. The image processing apparatus is arrangedfor processing a motion image or a series of still images in variousmanners and equipped with a controller as a computer and an interfacecircuit for communications. The peripheral apparatus is preferably apersonal computer accompanied integrally or separately with a displaymonitor for displaying the images. The peripheral apparatus is loadedwith software for acting as a member of the image processing system. Thesingle image processing apparatus may be connected to one or moreperipheral apparatuses. Two or more of the image processing apparatusesmay be connected over the computer network in the system.

Although the peripheral apparatus remains connected constantly to thecomputer network, it is preferably communicated with the imageprocessing apparatus under limited conditions. For example, each user isassigned with its password and can access to the system when providing alog-in command from its peripheral apparatus. If the access from theperipheral apparatus is delayed by a given length of time from the startof connection, it may be canceled.

A primary method of controlling the image processing system according tothe present invention is featured in which the image processingapparatus repeats a first step of generating an image to be processed, asecond step of generating a display image from the image to beprocessed, and a third step of saving the display image in a memory, andselectively performs, provided that an image transmission command basedon a stateless protocol is received from any peripheral apparatus, afourth step of extracting a desired number of the display images fromthe latest saved in the memory and a fifth step of generating andtransmitting to the peripheral apparatus a response of a specific formatto the image transmission command, predetermined by the image processingapparatus, which contains the extracted display images but not a controldata of the peripheral, and the peripheral apparatus performs a step Aof dispatching to the image processing apparatus the image transmissioncommand of a specific format predetermined by the peripheral apparatus,a step B of receiving the response to the image transmission commandfrom the image processing apparatus, and a step C of displaying thedisplay image, carried in the response, at a specific formatpredetermined by the peripheral apparatus on its monitor.

The image processing apparatus allows a user to perform a measurementaction for the purpose while monitoring the object of interest in amotion image or a series of still images. The display image is not onlythe image of the object to be measured but also an image of informationsuch as a modified image (e.g., binary image or edge enhanced image), aprofile of the result of measurement (e.g., histogram, graph, ornumerals), a data of the judgment for inspection, and a time table forthe processing action. The display image can be displayed on the monitorprovided separately or integrally with the image processing apparatus.The display images produced for a given length of the past time can besaved in a succession or at intervals of a desired number in the memory.

The stateless protocol employed in the present invention is designed forproducing and transmitting a response to a command to the destinationwhich has dispatched the command. The HTTP protocol is a typical exampleof the stateless protocol as used preferably.

When cycles of the communication at the stateless protocol are carriedout between two apparatuses, the apparatus at the response side cantransmit a response of information independently upon receiving eachcommand regardless of the preceding or succeeding contents of command.In a common computer system, every response to a command of thestateless protocol can be accepted. Since the present invention allowsthe communications between the image processing apparatus and eachperipheral apparatus to be conducted at the stateless protocol format,its controlling action over the communications can hardly be interruptedby any means for limiting the input of external data such as a firewallor a router.

At the fourth step in the image processing apparatus, the latest displayimage can be read out from the memory. When a number of the images aresaved in the memory, a desired succession of the images from the latestcan be obtained. A group of the images read out from the memory areallocated in a time sequence and carried on the response. Among a groupof the images, the earliest is extracted directly and the others may bemodified to differential images based on the previous image. In thatcase, the differential images may be treated as the display images. Thedisplay images can also be subjected to the compression and encryptionprocessing.

The peripheral apparatus according to the present invention is arrangedfor dispatching an image transmission command, receiving from the imageprocessing apparatus a response to the command which carries the latestdisplay image at the time, and displaying the latest image on itsmonitor screen. When the response carrying two or more display images isreceived, the peripheral apparatus can display the images in asuccession from the earliest.

The method according to the present invention may be modified, as afirst feature, in which the image processing apparatus performs thefourth and fifth steps whenever receiving the image transmission commandfrom the peripheral apparatus, and the peripheral apparatus repeats thestep A and the step B in this order and performs the step C whilerepeating the step B. More specifically, the peripheral apparatusdispatches a command for transmission of the image to the imageprocessing apparatus and upon receiving the response to the command,display the image before dispatching another image transmission command.Although the step C for displaying the image is carried out after thereceipt of the response is completed, it may be conducted in parallel tothe step of receiving the response (as will be eligible for thefollowing modifications).

When the processing steps are hastened in the peripheral apparatus, theimage can be displayed at almost the same time as in the imageprocessing apparatus. This allows the user at the remote location tocommunicate and cooperate smoothly with the user at the image processingapparatus while monitoring the same image. Also, the action and theresult of measurement in the image processing apparatus can be shared bythe two users at different locations.

Even if its speed of processing or communication is not speeded up, theperipheral apparatus can receive the latest display image at the momentwhen the receipt of the image is ready. Alternatively, the display imageto be transmitted is decimated in the size of data by the action of theimage processing apparatus to match the speed of processing orcommunication at the peripheral apparatus. In the latter case, thedisplay image may be displayed at a lower rate of definition (a joggedimage) on the monitor of the peripheral apparatus. As the display imageis lagged by a negligible time from that of the image processingapparatus, it permits the user at the peripheral apparatus toacknowledge the action o the image processing apparatus without error.

The response carrying the display image but not the controlling data forthe peripheral apparatus is produced at a desired form at predeterminedby the image processing apparatus. Also, the peripheral apparatus isassigned with desired formats for dispatching the image transmissioncommand and displaying the display image which are predetermined by theperipheral apparatus. The software installed in the peripheral apparatusis a specific one designed for the image processing system but not theWeb browser. It is true for handling the response transmitted from theimage processing apparatus with the Web browser to attach thecontrolling data to the data of the response in addition to the imagedata, hence increasing the overall data size of the response. This willcause the peripheral apparatus to retard its display action when thecontrolling data carried in the response is operated upon the receivingaction. It is particularly unfavorable when the display image is bulky.

The method according to the present invention may be modified, as asecond feature, in which the image processing apparatus performs thefourth and fifth steps whenever receiving the image transmission commandfrom the peripheral apparatus, and the peripheral apparatus repeats thestep A and the step B in order of B and A after executing the step A atleast one time and, as preceded by the step A, performs the step C whilerepeating the step B. After the image transmission command is dispatchedat least once and its response is received, another image transmissioncommand can be dispatched prior to the display of the display imagecarried in the response.

The second feature dispatches the image transmission command to theimage processing apparatus at the timing faster than the first feature.Accordingly, the peripheral apparatus can receive the display images atintervals of a shorter period than the first feature. Even if thedisplay image is fast at the image processing apparatus, its motion willbe followed by the corresponding image at the peripheral apparatus.

The method according to the present invention may be modified, as athird feature, in which the image processing apparatus waits for a givenperiod of time when receiving the image transmission command from oneperipheral apparatus and, when receiving the image transmission commandfrom another peripheral apparatus during the waiting period, transmitsthe common response to both the one and another peripheral apparatuses.This allows the image processing apparatus connected with two or moreperipheral apparatuses not to necessarily respond to each command fromthe peripheral apparatus and reduce its role for the controlling action.The common response to the peripheral apparatuses may be produced at thestart, the end, or just before the end of the waiting period. Thetransmission of the common response may be conducted by transmitting onedata to the peripheral apparatuses in a sequence.

The method according to the present invention may be modified, as afourth feature, in which the peripheral apparatus performs one cycle ofthe steps A, B, and C, determines from the execution of the cycle aprocessing period of time needed for communicating with the imageprocessing apparatus, and performs a step D of dispatching a secondaryimage transmission command including the processing period, and theimage processing apparatus performs the fourth and fifth steps one timein response to the image transmission command including non of theprocessing period and when receiving the secondary image transmissioncommand including the processing period, performs the fourth and fifthsteps whenever the processing period is elapsed. Also, the peripheralapparatus repeats the steps B and C after completing the step D.

In this feature, the processing period needed for communicating with theimage processing apparatus is equivalent to duration of time from thedispatching of the image transmission command to the receipt of theresponse or from the dispatching of the image transmission command tothe display of the image. The image processing apparatus when receivingthe image transmission command including the processing period generatesand transmits a response automatically when the processing period hasbeen elapsed. Accordingly, the peripheral apparatus can automaticallyreceive and display the latest image on its monitor screen withoutdispatching another image transmission command.

Alternatively in this feature, the peripheral apparatus allows the stepsB and C to be executed one cycle or repeated for a given period beforethe steps A, B, and C are conducted one cycle. Then, the step D isperformed according to the duration of time required for conducting theabove steps. The image processing apparatus when receiving another imagetransmission command including non of the processing period from theperipheral apparatus during its automatic action of generating andtransmitting the response, terminates the automatic action of generatingand transmitting the response and handles the secondary imagetransmission command. Accordingly, even when the duration of time neededfor the communication is varied by a busy period of the communicationnetwork or the like, it can be compensated. In particular, theprocessing period produced at the initial setting when the communicationnetwork is busy can be shortened after the business is eliminated. As aresult, the interval between two consecutive actions of receiving thedisplay images will be minimized.

It may also be possible that the peripheral apparatus measures a periodof time in which one cycle of the steps B and C is executed duringperforming the steps B and C after the step D and when the measuredperiod is shorter by a given length than the processing period, repeatsthe step D through updating the processing period with the measuredperiod. In this case, the image processing apparatus modifies theinterval between the fourth step and the fifth step in response to theupdated processing period. Accordingly, the processing period canreadily be updated when the display image is received at shorterinterval than the processing period at the initial setting.

The method according to the present invention may be modified, as afifth feature, in which the image processing apparatus repeats thefourth and fifth steps in response to the image transmission commandfrom the peripheral apparatus, and the peripheral apparatus performs thestep A one time and performs the steps B and C whenever receiving theresponse from the image processing apparatus.

In this feature, once the image transmission command is dispatched fromthe peripheral apparatus, the action of the image processing apparatusextracting the display image from the memory and generating andtransmitting the response to the command which includes the displayimage is repeated. Hence, when the peripheral apparatus is a computersized enough to follow the fourth and fifth steps, its action ofdisplaying the image can substantially be synchronized at the timingwith the display action of the image processing apparatus. This allowsthe user at the peripheral apparatus to monitor the display image atalmost the same timing as at the image processing apparatus.

It may also be possible as the fifth feature that the peripheralapparatus, when the receipt of the response is completed at the step B,performs an action of notifying the image processing apparatus of thecompletion of the receipt, and the image processing apparatusacknowledges the end of the fifth step from the notification of thereceipt completion from the peripheral apparatus and repeats the fourthstep.

This is feasible when the protocol is used of a lower level (such as TCPprotocol) than that of the stateless protocol where a message ofcompletion of the receipt is sent from a party which has received thedata. As the protocol divides the response data into packets andtransmits the succeeding packet upon receiving a receipt completionmessage from the peripheral apparatus, the receipt completion messageafter the transmission of the last packet acts as the receipt completionnotification in the above fifth feature.

Accordingly, the display images can be extracted and transmitted atintervals of time corresponding to the speed of processing orcommunication at the peripheral apparatus. Even when the speed ofprocessing or communication at the peripheral apparatus is not at ahigher rate, the display can be updated as timed with that of the imageprocessing apparatus.

The method according to the present invention may be modified, as asixth feature, in which the peripheral apparatus, when the settingaction for the image processing apparatus is conducted at its operationpanel, performs a step E of generating and dispatching an settingacceptance command including the setting data to the image processingapparatus, and the image processing apparatus when receiving the settingacceptance command, performs a sixth step of updating the display imagewith the setting data carried in the command and a seventh step ofsaving the updated display image in the memory.

The peripheral apparatus conducts the action of examining whether thesetting action is made or not in parallel with the steps A, B, and C andperforms the step E depending on the setting action. The imageprocessing apparatus when receiving the setting acceptance commandproduced at the step E generates the latest display image including thesetting data (type of the setting action, object of the setting action,and parameters and measurements of the settings) and can thus update thedisplay of the image. As the updated display image is saved in thememory of the image processing apparatus, it can be received by theperipheral apparatus dispatching the image transmission command afterthe setting acceptance command.

It may be possible, as the sixth feature, in which the setting actionincludes selecting items for the setting, and the image processingapparatus generates at the sixth step the display image including a menuof the items for the setting when receiving the setting acceptancecommand including the setting data.

The setting action may also include moving the cursor to a selected itemof the menu and clicking the cursor at the selected item. Throughconducting the setting action, the conditions of the menu display on thescreen at the image processing apparatus can be updated (for example,cursor movement, cursor inversion, another menu, end of menu).Simultaneously, the display screen at the peripheral apparatus can beupdated.

This feature allows the image on the monitor screen of the imageprocessing apparatus to be modified and updated by the setting action ofthe user at the remote location. Also, the updated image is saved in thememory and can be transmitted together with the display data to theperipheral without delay. Accordingly, the setting data on the sameimage screen can be shared by two users at the manufacturing site andthe remote location respectively. The action of the image processingapparatus in response to the setting acceptance command is not limitedto the displaying action. For example, when one item in the setting datais entered, its relevant action can be made. When the action of readinga data saved in the image processing apparatus is requested, the datacan readily be retrieved and transmitted to the requesting peripheralapparatus.

An image processing system is provided to which the foregoing method ofthe present invention is applied, as featured in that the imageprocessing apparatus includes an image inputting means for inputting animage to be processed, a display image generating means for generating adisplay image from the image input by the image inputting means, amemory for saving the display image, a command receiving means forreceiving an image transmission command based on a stateless protocolfrom any peripheral apparatus, an image extracting means for extractinga desired number of the display images from the latest saved in thememory, a responding means for generating and transmitting to theperipheral apparatus a response of a specific format to the imagetransmission command, predetermined by the image processing apparatus,which contains the extracted display images but not a control data ofthe peripheral apparatus, and a controlling means for actuating theimage extracting means and the responding means, provided that thecommand receiving means receives the image transmission command, and theperipheral apparatus includes a command dispatching means fordispatching to the image processing apparatus the image transmissioncommand of a specific format predetermined by the peripheral apparatus,a response receiving means for receiving the response to the imagetransmission command from the image processing apparatus, and a displaycontrolling means for displaying the display image, carried in theresponse, at a specific format predetermined by the peripheralapparatus.

The image inputting means of the processing apparatus in the systemincludes an interface circuit and an A/D converter circuit for inputtingand processing the image, whereby the image recorded by a camera isacquired and converted into a digital form which is then subjected tothe measurement action. If the imaging means is a digital camera, theA/D converter circuit may be eliminated.

The display image generating means, the command receiving means, theimage extracting means, the responding means, and the controlling meansall are implemented by a computer hardware having correspondingprograms. The display image generating means and the image extractingmeans maybe provided in a dedicated hardware.

The display image generating means is arranged for generating the imageto be displayed from the image received by the image inputting means, amodification of the image (binary image or edge enhanced image), theresult of measurement from the input image or the modified image, andtheir two or more combination. Also, the display image can be saved in amemory by the display image generating means. The memory may be sizedfor saving one or preferably more display images.

The command receiving means is arranged for receiving a command from theperipheral apparatus via a communication interface connected with thecomputer network. The responding means is also arranged for transmittinga response to the peripheral apparatus via the communication interface.

The command dispatching means, the response receiving means, and thedisplay controlling means in the peripheral apparatus are implemented bycorresponding programs installed in the computer as a main control unitin the peripheral apparatus for performing the steps A, B, and C.

The system of the first feature of the present invention may be operatedin which the controlling means in the image processing apparatusactuates the image extracting means and the responding means wheneverthe command receiving means receives the image transmission command fromthe peripheral apparatus, and the command dispatching means in theperipheral apparatus dispatches another image transmission commandwhenever the receiving means receives the response and the displaycontrolling means displays the display image.

The system of the second feature of the present invention may beoperated in which the controlling means in the image processingapparatus actuates the image extracting means and the responding meanswhenever the command receiving means receives the image transmissioncommand from the peripheral apparatus, and the command dispatching meansin the peripheral apparatus conducts the dispatching of the imagetransmission command at least one time and dispatches another imagetransmission command prior to the display controlling means displayingthe display image, whenever the receiving means receives the response.

The system of the third feature of the present invention may be operatedin which the controlling means in the image processing apparatus directsthe responding means to wait for a given period of time when the commandreceiving means receives the image transmission command from oneperipheral apparatus and, when receiving the image transmission commandfrom another peripheral apparatus during the waiting period, transmitsthe common response to both the one and another peripheral apparatuses.Alternatively, the image extracting means may also be controlled to waitas the responding means does and carry out its image extracting actionjust before the common response is transmitted.

The system of the fourth feature of the present invention may bemodified in which the peripheral apparatus includes a transmissioncommand controlling means for performing one cycle of the actions of thecommand dispatching means, the response receiving means, and the displaycontrolling means, determining from the execution of the cycle aprocessing period of time needed for communicating with the imageprocessing apparatus, and directing the command dispatching means todispatch a secondary image transmission command including the processingperiod, and the controlling means in the image processing apparatusdirects the image extracting means and the responding means to performtheir action one time in response to the image transmission commandincluding non of the processing period and when receiving the secondaryimage transmission command including the processing period, directs theimage extracting means and the responding means to perform their actionwhenever the processing period is elapsed. The dispatch commandcontrolling means in the peripheral apparatus may be implemented by aprogram for controlling its action.

The system of the fourth feature may further be modified in which thetransmission command controlling means in the peripheral apparatusincludes a measuring means for measuring a period of time in which onecycle of the actions of the response receiving means and the displaycontrolling means is executed when the secondary image transmissioncommand including the processing period has been received, and aprocessing period updating means for updating the processing period withthe measured period when the measured period is shorter by a givenlength than the processing period and then directs the commanddispatching means to dispatch a further image transmission commandincluding the updated processing period, and the controlling means inthe image processing apparatus when receiving the further imagetransmission command including the updated processing period, modifiesthe interval between the action of the image extracting means and theaction of the responding means according to the updated processingperiod.

The system of the fifth feature of the present invention may be operatedin which the controlling means in the image processing apparatus directsthe image extracting means and the responding means to repeat theiraction in response to the image transmission command received by thecommand receiving means, and the peripheral apparatus includes adispatch controlling means for directing the command dispatching meansto dispatch the image transmission command one time and then actuatingthe response receiving means and the display controlling means wheneverthe response is received from the image processing apparatus. Thedispatch controlling means may be implemented by a program.

The system of the fifth feature may further be modified in which thedispatch controlling means in the peripheral apparatus, when the receiptof the response by the response receiving means is completed, performsan action of notifying the image processing apparatus of the completionof the receipt, and the controlling means in the image processingapparatus includes a means for receiving the notification of the receiptcompletion from the peripheral apparatus and acknowledges the end of theaction of the responding means from the notification of the receiptcompletion and then restarts the action of the image extracting means.

The system of the sixth feature of the present invention may be modifiedin which the peripheral apparatus includes an inputting means forinputting the setting data for the image processing means and an settingacceptance command dispatching means for, when the setting data for theimage processing apparatus is input by the setting action of theinputting means, generating and dispatching an setting acceptancecommand including the setting data to the image processing apparatus,and the display image generating means in the image processingapparatus, when the setting acceptance command is received by thereceiving means, updates the display image with the setting data carriedin the command and saves the updated display image in the memory.

The inputting means may be implemented by a computer having a programfor providing a desired form of the setting screen on the monitor. Thesetting acceptance command dispatching means may be implemented by thecomputer having a program for performing the step E. As the settingacceptance command requires no response from the image processingapparatus, it needs not to be in the form of a stateless protocol suchas the image transmission command. Alternatively, the setting acceptancecommand may be generated of the stateless protocol format so that thedisplay image carried in the response from the image processingapparatus is accompanied with details of the setting data.

The image processing system according to the present invention may bemodified, as a more preferable form, in which the setting actionincludes selecting items for the setting, and the display imagegenerating means in the image processing apparatus generates the displayimage including a menu of the items for the setting when receiving thesetting acceptance command including the setting data. This allows thesame menu to be displayed on both the monitor of the image processingapparatus and the monitor of the peripheral apparatus for ease ofacknowledging the setting data.

The image processing apparatus in the image processing system may beconnected with a console, a monitor, and any other apparatuses as wellas the imaging means. Alternatively, those apparatuses may be assembledintegral with the image processing apparatus.

The peripheral apparatus is preferably, but not limited to, a personalcomputer and may be a mobile device (e.g., PDA) having an imagedisplaying function.

A program according to the present invention is provided for functioninga computer as a peripheral apparatus connected over a computer networkto an image processing apparatus. The program may be saved in anavailable storage medium such as CD-ROM or DVD and distributed over thecomputer network.

The program according to the present invention may be modified in whichthe computer is assigned with a command dispatching means fordispatching to the image processing apparatus an image transmissioncommand at a specific format or stateless protocol format predeterminedby the peripheral apparatus, a response receiving means for receiving aresponse to the image transmission command from the image processingapparatus, and a display controlling means for displaying a displayimage, carried in the response, at a specific format predetermined bythe peripheral apparatus on its monitor screen. Alternatively, theprogram may include a program for determining the setting on theperipheral apparatus to match each modification of the image processingsystem.

The present invention allows the communications between the imageprocessing apparatus and the peripheral apparatus(es) to be conductedregardless of the network environments with the use of the statelessprotocol format. Also, since the image produced by the image processingapparatus is displayed substantially at the same timing on theperipheral apparatus, it can be monitored substantially at the same timeby a user at the manufacturing site and a user at the remote location,hence ensuring the communications between the two users with smoothness.Moreover, as the understanding of the controlling, measurement, andother actions of the image processing apparatus is shaped by the usersat different locations, the image processing system can be improved inthe expediency.

The present invention allows the image to be modified by the settingaction of the peripheral apparatus and simultaneously displayed at boththe image processing apparatus and the peripheral apparatus.Accordingly, the image modified by the setting action of the peripheralapparatus can concurrently be monitored by the users at two differentlocations. Particularly, the user at the manufacturing site canacknowledge details of the setting action conducted by the user at theremote location and thus carry out its processing action without erroreven if no one skilled in the operation of the image processingapparatus is present at the site. Also, the knowledge of the skilleduser at the remote location can smoothly be transferred to themanufacturing site.

The present invention allows the image processing apparatus to transmitto the peripheral apparatus the response which is produced at a specificformat predetermined by the image processing apparatus and carries nocontrol data for the peripheral apparatus while the peripheral apparatusproduces an image transmission command and displays the display imagecarried in the response at a specific format predetermined by theperipheral apparatus. Accordingly, the display screen at the peripheralapparatus can be updated for modifying the setting data and improvingthe setting action by replacement of the software in the peripheralapparatus which requires no real-time controlling action, but not of thesoftware in the image processing apparatus. It is hence totallyunnecessary to conduct such troublesome tasks as removal and resettingof the saved data which are commonly executed for updating the softwarein the image processing apparatus. Also, any fault, such ascommunication timing error or measurement discrepancy, which normallyoccurs during the real-time controlling action due to instability in thematching with undated software can be avoided.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 schematically shows an image processing system according to thepresent invention.

FIG. 2 shows another arrangement of the image processing system.

FIG. 3 shows a block diagram showing the arrangement of an imageprocessing apparatus.

FIG. 4 shows a block diagram showing the functional arrangement of aperipheral.

FIG. 5 shows an explanatory illustration of a structure of response datatransmitted from the image processing apparatus to the peripheral.

FIG. 6 shows an explanatory illustration showing the exchanging of TCPbased data between the image processing apparatus and the peripheral.

FIG. 7 shows an explanatory illustration showing steps of thecommunication controlling action between the image processing apparatusand the peripherals.

FIG. 8 shows an explanatory illustration showing steps of thecommunication controlling action between the image processing apparatusand the peripherals.

FIG. 9 shows an explanatory illustration showing steps of thecommunication controlling action between the image processing apparatusand the peripheral.

FIG. 10 shows an explanatory illustration showing steps of thecommunication controlling action between the image processing apparatusand the peripheral.

FIG. 11 shows an explanatory illustration showing steps of thecommunication controlling action between the image processing apparatusand the peripheral.

FIG. 12 shows a flowchart illustrating the action of the imageprocessing apparatus in response to the steps shown in FIG. 11.

FIG. 13 shows an explanatory illustration showing steps of thecommunication controlling action between the image processing apparatusand the peripherals.

FIG. 14 shows an illustration showing a structure of the imagetransmission command.

FIG. 15 shows an illustration showing a structure of the response.

FIG. 16 shows a schematic view showing a drawback when the response iscached into a proxy server.

FIG. 17 shows a schematic view showing the communication executed by thesteps shown in FIGS. 10 and 11.

FIG. 18 shows display screens in the relationship between the imageprocessing apparatus and the peripheral.

FIG. 19 shows display screens in the relationship between the imageprocessing apparatus and the peripheral.

FIG. 20 shows a flowchart showing steps of the controlling action in therelationship between the image processing apparatus and the peripheral.

FIG. 21 shows a flowchart showing steps of the controlling action in therelationship between the image processing apparatus and the peripheral.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1 schematically illustrates an image processing system according toone embodiment of the present invention.

The image processing system consists mainly of an image processingapparatus 1 provided in a manufacturing site (such as an industrialfactory) and peripheral apparatuses 2 provided at (remote) locationsdistanced from the manufacturing site which are connected by theInternet 3. The peripheral apparatus 2 used for accessing is commonly apersonal computer (referred to as a peripheral 2 hereinafter). While asingle unit of the image processing apparatus 1 is provided, two or moreperipherals 2 (2A, 2B, 2C, and so on in the figure) can be connected inthe system.

The image processing apparatus 1 includes an interface circuit for theEthernet (a registered trade name) which can connect with the Internet 3via a router (not shown). The peripherals 2 may be assigned to anothersystem such as a LAN or the Intranet. Also, a means for controlling theaccess to data from the outside, such as a firewall 4 or a router 5, maybe provided across a connection between the Internet 3 and theperipheral, e.g., 2B or 2C, as shown.

FIG. 2 illustrates another arrangement of the image processing system.The another arrangement includes two or more image processingapparatuses 1 (two 1A and 1B in the figure). Both the image processingapparatuses 1A and 1B like that of the embodiment shown in FIG. 1 areconnected to the Internet 3. Each of the peripherals 2 can selectivelyconnect to one or both of the image processing apparatuses 1A and 1B.The image processing apparatuses 1A and 1B may be located not only inthe same manufacturing site but also in different sites respectively.

In FIGS. 1 and 2, the network connected to the computer is not limitedto the Internet 3 but may be any user dedicated network which employsthe communication protocols (e.g., HTTP or TCP/IP) used over theInternet 3.

One embodiment of the present invention applicable to the system shownin FIG. 1 or 2 will now be described.

The image processing apparatus 1 comprises a set of components installedin a box-like housing 10 as shown in FIG. 3. The housing 10 is connectedto a television camera 11 (referred to as a camera 11 hereinafter) and aconsole 12, thus forming a fundamental arrangement. Alternatively, theimage processing apparatus 1 may be connected with other personalcomputers (which are different from the terminal 2 shown in FIG. 1 or 2and allocated at the same manufacturing site as of the image processingapparatus 1) or a PLC (programmable logic controller), an upper levelinstrument such as a PLC (programmable logic controller), and a monitor.

FIG. 3 is a block diagram of the image processing apparatus 1. The imageprocessing apparatus I includes a controller 10 a comprising CPU 101, anSDRAM 102, and a flash memory 103. The CPU 101 is connected to anRS-232C based interface 109 and via a bus 104 to an image processor 105,an input interface 106, an Ethernet (registered trade name) interface107, and a parallel interface 108. The image processor 105 is connectedwith a camera interface 110, a monitor interface 111, an image memory112, and a model memory 113.

The camera interface 110 is provided for receiving a video signal fromthe camera 11. The image processor 105 is an image processing dedicatedLSI including a binary circuit, an edge extractor circuit, and acorrelation operator circuit and arranged for processing at high speedsthe image received from the camera interface 110. Also, the imageprocessor 105 includes a function for generating an image to bedisplayed (e.g., an image mixing circuit). The image to be displayed istransferred via the monitor interface 111 to a monitor (not shown).

The image memory 112 has areas assigned for storage of the data ofdisplay images and input images received from the camera interface 110.The model memory 113 is provided for storage of model images used as themodels for the action of pattern matching process of the correlationoperator circuit. The SDRAM 102 is used as a work memory for the CPU101. The flash memory 103 is provided for storage of programs foroperating the CPU 101 and parameters for the image processing. The flashmemory 103 has areas assigned for storage of the images received in thepast for measurement and the results of the measurement.

The input interface 106 is provided for receiving an input signal fromthe console 12. The parallel interface 108 and the RS232C basedinterface 109 may be connected to upper level peripherals (e.g., a PLCand personal computers provided at the manufacturing site as describedpreviously) and their data to be passed are different in the format fromeach other. The RS232C based interface 109 is provided for exchangingmainly numerical data. The parallel interface 108 is provided forreceiving commands from the upper level instruments and transmittingstate signals to the same.

The Ethernet (registered trade name) interface 107 (referred to as aninterface 107 hereinafter) is provided for communication over theInternet 3. In response to a command for transmission of image data,which will be explained later in more detail, the CPU 101 retrieves theimage to be displayed from the image processor 105 and generates aresponse including the image to be displayed which is then transmittedfrom the interface 107. The response includes no data for controllingthe action of display or data exchange for the peripherals and itsformat may be modified as desired. The format of the response may bepredetermined prior to the image processing action by a desired settingscheme.

In action, the image processor 105 receives a digital form of image ofinterest via the camera interface 110 and carries out its action ofprocessing the received image. The action includes, for example,extracting the edge data from the received image and measuring itscoordinate points and gray scale direction. Also, the action includes astep of pattern matching between the received image and the model savedin the model memory 113 to determine the location of an object in theimage.

The result of measurement is transferred from the image processor 105 tothe CPU 101. The CPU 101 repeats an action of further detailedmeasurement or judging from the measurement whether the object is goodor not. The result of measurement may be transferred via the interfaces108 and 109 to external instruments or saved in the flash memory 103.

Simultaneously, the image processor 105 when receiving a command fromthe CPU 101 generates an image to be displayed on the monitor. Theformat of the display image may arbitrarily be determined or preferablypredetermined prior to the image processing action by a desired settingscheme. The image to be displayed may be one of input image or amodification (binary image or edge image) of the image generated duringthe image processing action. Alternatively, the display image may be ahistogram or a graphic profile of the measurement or a combination ofthe image and an indicator such as a pointer for marking the extractionlocation of the object. Moreover, a menu of setting items may bedisplayed for ease of determining the settings including the parametersand the mode of action.

The remove peripheral 2 is equipped in advance with software for theimage processing action. The software allows the peripheral 2 to fetchthe display image from the image processing apparatus 1 at the timingsubstantially equal to the display of the image and display the same ata desired format on its monitor before starting the image processingaction. The format of the display image can arbitrarily be modified atthe peripheral. At the remote peripheral 2, the action of determiningthe settings on the image processing apparatus 1 or downloading themeasurement or history of the actions saved in the image processingapparatus 1 can be conducted with entry of peripheral2 (keyboard ormouse etc.).

FIG. 4 illustrates a functional arrangement of the peripheral 2controlled by the dedicated software. The program of the software isinstalled directly from a given storage medium or downloaded from theInternet 3 before saved in the hard disk drive of the peripheral 2.

As shown, an entry 21 is provided for receiving commands and settingsdesired by the user. A communication processor 22 is provided forexchanging data with the image processing apparatus 1. A data decoder 23is provided for converting the data received from the image processingapparatus 1 into a feasible format. This function is needed because thedata received from the image processing apparatus 1 is commonlycompressed or encrypted. An image display 24 is provided for extractinga display image from the decoded data and displaying the display imageon its monitor screen.

A memory controller 25 is also provided for reading or writing data on amemory of the peripheral 2 for executing the communications. Forexample, the display image data or measurement data received by thecommunication processor 22 may be saved or the data for transmission tothe image processing apparatus 1 may be registered in advance.

The communication processor 22 in the embodiment supplies the imageprocessing apparatus 1 with a command of the HTTP protocol format as thepredetermined setting format prior to the image processing action andreceives a response of the same format to obtain the image to bedisplayed. Because the HTTP protocol is a stateless protocol, it can notbe interrupted by the firewall 4 or the router 5 which limits thereception of external data. The format may arbitrarily be modified asdesired.

In the embodiment, the display image produced by the image processingapparatus 1 is decimated depending on the rate of the processing actionat the peripheral 2 or the speed of data transmission so that its updateversion is carried on the data at each transmission. This allows theuser at the peripheral 2 to monitor the same display image as viewed bythe user at the image processing apparatus 1 with substantially no timelag.

FIG. 5 illustrates a series of steps for transmitting the data from theimage processing apparatus 1 to the peripheral 2. The data produced bythe image processing apparatus 1 has a header coupled to the body ofinformation including the (display) data to be transmitted at the HTTPprotocol. The header contains a message indicative that the protocol isof the HTTP format and a type of the data. The header will be referredas an HTTP header hereinafter.

The data of the HTTP protocol is divided into packets P1, P2, . . . andPn of the TCP format. Each of the packets has a header carrying the portnumbers of the transmitter and the receiver as well as the sequencenumber of its packet. The header will then be referred to as a TCPheader hereinafter. Also in the figure, the TCP header is followed bydata segments D1, D2, . . . and Dn to be transmitted.

The HTTP header is attached to only the first packet P1.

When receiving the data segments D1, D2, . . . and Dn, their TCP headersare examined by the firewall 4 or the router 5. Simultaneously, the HTTPheader attached to the first data D1 is also examined. When the HTTPheader contains the message of responding to the command from theperipheral 2, the firewall 4 or the router 5 judges that the datasegment D1 is an eligible data to be passed and attaches a given headerto the data segment D1 for entrance into the system. Then, thesucceeding data segments D2 to Dn are accepted as eligible data throughexamining the port number and the sequence number in the TCP header andattached with given headers for entrance into the system.

According to the TCP protocol, the TCP header once received is providedwith a message indicative of the receipt of the data has been completedand returned back from the receiver computer to the transmitter computerso that the transmission and receipt of the data is acknowledged. Thisaction will be referred to as receipt completion message hereinafter.

FIG. 6 illustrates the transmission of the data shown in FIG. 5 from theimage processing apparatus land the transmission of the message from theperipheral 2 at the TCP protocol. As explained, the n packets of thedata are transmitted from the image processing apparatus 1. The imageprocessing apparatus 1 waits for receiving the receipt completionmessage from the peripheral 2 after the transmission of each packet ofthe data. The next packet of the data is transmitted only when thereceipt completion message is received. Accordingly, the transmission ofthe data can be made depending on the rate of the processing action atthe peripheral 2 or the speed of the transmission.

FIGS. 7 to 13 illustrate different actions of controlling thecommunications between the image processing apparatus 1 and theperipheral 2 (including an action of controlling the peripheral 2 forimage display). Each action starts with a size of image data at the HTTPprotocol transmitted from the image processing apparatus 1 to theperipheral 2. In any case, the exchange of data shown in FIG. 6 isrepeated several times between the image processing apparatus 1 and theperipheral 2 to complete one cycle of the transmission of the HTTP data.

In FIGS. 7, 8, and 13, the single image processing apparatus 1 isconnected with two of the peripherals 2A and 2B. In FIGS. 9, 10, and 11,the single image processing apparatus 1 is connected to the singleperipheral 2. As shown, the steps of action at the peripheral 2 aredenoted by the alphabetical letters A, B, C, and so on while the stepsof action at the image processing apparatus 1 are denoted by thenumerals 1, 2, 3, and so on. Like steps are denoted by like letters ornumerals throughout the figures. The following description is hence madeusing the alphabetical letters and the numerals.

FIG. 7 illustrates the dispatching of a command for receiving the imageto be displayed (referred to as an image transmission command) from theperipheral 2 to the image processing apparatus 1 (Step A). Whenreceiving the image transmission command, the CPU 101 in the imageprocessing apparatus 1 drives the image processor 105 and the interface107 to retrieve the image to be displayed from the image memory 112 andtransmit a respond including the display image to the peripheral 2(Steps 1, 2, and 3). Meanwhile, the display image is subjected tocompression and encryption process. The display image may be at leastone of the latest images or a set of images sequenced from the newest.

Upon receiving the response to its image transmission command from theimage processing apparatus 1, the peripheral 2 display the display image(Step B). The peripheral 2 repeats the dispatch of the imagetransmission command, the receipt of the response, and the display ofthe image to be displayed. The image transmission command is generatedat the HTTP protocol as described previously. The image processingapparatus 1 performs its action in response to the image transmissioncommand independently regardless of the preceding or succeeding commandor the content of the command. The peripheral 2 can hence produce animage transmission command at desired intervals of time according to itsprocessing capability to obtain and display the latest image to bedisplayed. Since the current computers and networks are enhanced in thespeed of data transmission, the time lag between the generation of animage to be displayed in the image processing apparatus 1 and thedisplay of the display image in the peripheral 2 will be as short asnegligible. This allows any user at the remote location to monitor theimage at substantially the same timing as at the manufacturing site.

FIG. 8 illustrates the transmission of a response to the imagetransmission command from the image processing apparatus 1 to theperipheral 2. In this action, the image processing apparatus 1 waitsuntil a predetermined length of time is elapsed from the receipt of theimage transmission command (Step 4). During the length of waiting time,no response to the image transmission command from any other peripheral2 is made. After the predetermined length of time is elapsed, the imageprocessing apparatus 1 generates and transmits at once the responses tothe first image transmission command from the peripheral 2 and the otherimage transmission commands from the other peripherals 2 (Step 5).

Also in this action, a common response to the peripherals 2 can begenerated by the image processing apparatus 1 of which the controllingaction is thus eased. Particularly, the response is transmitted to theperipheral 2 which dispatches the first image transmission command andthe other peripherals 2 which dispatch their image transmission commandsduring the waiting time. Accordingly, even if the dispatch of the imagetransmission command from, e.g., the peripheral B is delayed, it willnever affect or retard the updating of the images to be displayed in theother peripherals 2. Also, the peripheral B which delays the dispatch ofthe image transmission command can certainly receive the response afterthe waiting time thus having no error in the displaying of the image.

In the action shown in FIGS. 7 and 8, the peripheral 2 has to repeat theaction of dispatching the image transmission command. FIG. 9 illustratesthe peripheral 2 dispatching the image transmission commandfundamentally two times. After dispatching the first image transmissioncommand (Step B), the peripheral 2 measures the duration of time fromthe dispatch of the image transmission command via the receipt of theresponse the image processing apparatus 1 to the completion of thedisplay of the image. Then, the period of time needed for processingbefore the receipt of the image to be displayed is calculated from themeasurement (Step D) and the second image transmission command isdispatched including the calculated processing period (Step C).

The image processing apparatus 1 performs the responsive action oncewhen receiving the first image transmission command which carries non ofthe processing period as equal to that shown in FIGS. 7 and 8. Whenreceiving the second image transmission command which includes theprocessing period, the image processing apparatus 1 performs thedirectly responsive action and repeats the action of generating aresponse after the processing time is elapsed and transmits the responseto the peripheral 2 (Step 3A). After dispatching the second imagetransmission command, the peripheral 2 waits for receiving a not-directresponse from the image processing apparatus 1 and updates the displayof the image whenever receiving the response (Step B).

In the above action, any not-direct response (Step 3A) other than thedirect response to the image transmission command includes the HTTPheader which is identical to that attached to the directly response tothe image transmission command. Accordingly, the not-direct response canbe passed across the firewall 4 or the router 5 to the peripheral 2. Inthis action, the latest image to be displayed can be transmittedwhenever the processing time has been elapsed.

In FIG. 9, the image processing apparatus 1 is connected to the singleperipheral 2. When two or more of the peripherals 2 are connected, theprocessing time is examined at every peripheral 2 before the not-directresponse is transmitted (Step 3A).

FIGS. 10 and 11 illustrate the dispatching of the image transmissioncommand and the transmission of the response schematically denoted bythe arrows while the step B (of receiving and displaying the image)shown in FIGS. 7 to 9 is separated into two sub steps E and F.

The peripheral 2 shown in FIG. 10 has dispatched several imagetransmission commands (two times in the figure) following the firstimage transmission command before receiving the response to the firstimage transmission command. Then, when the action of receiving theresponse to the preceding image transmission command is completed (StepE), the dispatching of the next image transmission command is carriedout before the action of displaying the image starts (Step F). As thedispatching of the next image transmission command is conducted beforethe action of displaying the image received in response to the precedingimage transmission command, the interval of time between two consecutivereceipts of the image can be minimized. Accordingly, even when thedisplay action of the image processing apparatus 1 is fast, it can beresponded by the peripheral 2 without delay. When the image processingapparatus 1 shown in FIG. 10 is connected to two or more of theperipherals 2, it can simultaneously perform the action of respondingtwo or more image transmission commands received within a period of timesimilar to the action shown in FIG. 8.

FIG. 11 illustrates the peripheral 2 dispatching the image transmissioncommand once. In turn, the image processing apparatus 1 generates aresponse (Step 2) and transmits the same (Step 3). It also generates anot-direct response (Step 2) and repeats the action of transmission(Step 3A). This action is equivalent to the action in response to theimage transmission command with the processing time shown in FIG. 9.

The responsive action shown in FIG. 9 includes the steps 2 and 3A spacedby an interval corresponding to the processing time. However, theresponsive action shown in FIG. 11 is carried out using the controllingaction at the TCP format shown in FIG. 6. The controlling action will beexplained in more detail referring to FIG. 12.

The controlling action shown in FIG. 12 starts upon the receipt of theimage transmission command from the peripheral 2. At the first step A,the HTTP header is attached to the body including the latest image to bedisplayed to construct a response data. This is followed by Step B wherethe response data is divided into packets. The TCP header is attached toeach packet which is then transmitted (Step C). When the receiptcompletion message is received, the transmission of the packet isrepeated (Step D). Steps A to D are equivalent to the action shown inFIG. 6. In other words, each responsive action in the foregoingdescription comprises Steps A to D.

The action returns to Step A for generating another response when it isjudged “yes” at Steps D and E after the receipt completion message forthe last packet is received. Similarly, the repeating of Steps A to Eallows each response to be generated and transmitted upon the peripheral2 completing the receipt of the preceding response.

At the controlling action, the peripheral 2 can receive the images to bedisplayed from the image processing apparatus 1 at intervals of timedetermined depending on the rate of its processing action and the speedof the transmission. Since the response includes the latest image to bedisplayed at the time, it allows the peripheral 2 to display the imageat substantially the same timing as of the image processing apparatus 1.

FIG. 13 illustrates the same controlling action as shown in FIG. 11 withtwo or more of the peripherals 2 (two A and B in the figure). Thecontrolling action at the HTTP format (Step 1 in the figure) and thecontrolling action at the PCT format (Step 2 in the figure) arecorrelated. In the action, the first responsive action to the imagetransmission command is carried out at the timing of receiving the imagetransmission command. Then, the responsive action is conducted totransmit the common response to both the peripherals A and B.

The TCP format controlling action involves, as shown in FIG. 13(2),waiting for receiving the receipt completion message from each of theperipherals 2A and 2B after every transmitting action and repeats thetransmitting action only when a given number of the receipt completionmessages have been received. This may cause the transmitting action tobe repeated depending on the processing speed of the slowest peripheral2. However, as the transmitting action is carried out at once for allthe peripherals 2, its efficiency can remain not declined. It ispreferable to carry out the transmitting action for the peripherals 2excluding the slowest peripheral 2 which fails to dispatch the receiptcompletion message after a predetermined length of time is elapsed.

The structure of each data (at the HTTP protocol format) used in theactions will now be described. FIG. 14 illustrates a data structure ofthe image transmission command dispatched from the peripheral 2. Theimage transmission command consists of two lines of message. The messageP at the first line is to command the transmission of the image to bedisplayed. The message Q at the second line instructs not to cache theresponse to the image transmission command in a proxy server. As notshown, the messages P and Q are carried in the body and assigned with adedicated header.

FIG. 15 illustrates an example of the response of the image processingapparatus 1 to the image transmission command shown in FIG. 14. Theresponse consists of a header at the HTTP protocol format and a body.The header contains a message R at the first line indicative ofreceiving of the image transmission command. Carried at the third lineis the message Q in the image transmission command.

The body is divided into segments called as chunks. One chunk carries acombination of a data S indicating the size and a data T indicating thecontent. A group of the data combinations S and T represent one or moreimages to be displayed. When two or more images are to be carried in thebody, the earliest image is a reference image and the remaining imagesmay be expressed by differences from the reference image.

FIG. 16 illustrates an action of transmitting the image transmissioncommand and its response excluding the message Q. As shown, a proxyserver 6 is provided at the entrance or exit of the computer system atthe peripheral 2 for connection to the Internet 3.

The first image transmission command is dispatched at the time t fromthe peripheral 2, passed through the proxy server 6, and received by theimage processing apparatus 1 where a response to the command isgenerated. However, the response generated at the time t is cached intothe proxy server 6. The proxy server 6 is designed for, when saving acache data responding to the command, releasing the cache data but nottransferring the command. Accordingly, even when the second imagetransmission command is dispatched after a length a of time is elapsedfrom the time t, it is not received by the image processing apparatus 1and the past response generated at the time t is transmitted. This willprevent the peripheral 2 from updating the display of the image.

FIG. 17 illustrates an action of transmitting the image transmissioncommand and its response including the message Q. In this action, theresponse to the first image transmission command dispatched at the timet is not cached into the proxy server 6 but transferred to theperipheral 2. The same action as at the time t is repeated in responseto the image transmission command dispatched when the length a of timehas been elapsed from the time t as followed by transmitting theresponse generated currently with the image processing apparatus 1.Accordingly, when the image transmission command is repeatedlydispatched from the peripheral 2 as shown in FIGS. 7 to 10, the latestimage to be displayed can always be received by the peripheral 2 whichthus carries out the updating of the image with no failure.

The action of controlling the display with the setting determined by theperipheral 2 will be described. For determining the setting with theentry 21 shown in FIG. 4, a simulated image of the console 12 in theimage processing apparatus 1 (referred to as a simulated commanderhereinafter) is generated and displayed on the monitor of the peripheral2. The setting can thus be determined in the same sequence as at themanufacturing site by the user clicking desired buttons of the simulatedcommander with the use of a mouse. As the simulated commander isoperated, the communication processor 22 dispatches a command foraccepting the setting action (a setting acceptance command) to the imageprocessing apparatus 1. The setting acceptance command carries a dataabout details of the setting action. The image processing apparatus 1then generates a display image including a menu of the setting itemsfrom the data carried by the setting acceptance command for updating thedisplay. The display image is saved in the image memory 112 beforetransmitted to the peripheral 2 as the response to an image transmissioncommand which follows the setting acceptance command. This allows theperipheral 2 to update the display image as equal to in the imageprocessing apparatus 1.

ig. 18 illustrates display screens in the relationship between the imageprocessing apparatus 1 and the peripheral 2. This relationship standswhen the setting action is started by the peripheral 2. Displayed on adisplay screen 100 of the image processing apparatus 1 are a menu 8 ofthe setting items as well as an image 7 of the object to be measured. Adisplay screen 200 of the peripheral 2 provides a window 26 fordisplaying the image received from the image processing apparatus 1 anda simulated commander 27. The window 26 displays the same display imageas on the screen 100 of the image processing apparatus 1 including theimage 7 and the menu 8.

When the image processing apparatus 1 is connected with two or more ofthe peripherals 2, the display image including the menu 8 may bedisplayed on the other peripherals 2 which remain not operated for thesetting. Also, the display image displayed on the other peripherals 2may exhibit information about the current status of each user. When theperipheral 2 commands the receipt of desired data, the image processingapparatus 1 can retrieve the desired data determined by the settingacceptance command and transmit it as the response to the terminal 2.Accordingly, each user at the peripheral 2 can receive its desired datasuch as a history of the measurement through downloading.

FIG. 19 illustrates display screens when the image processing apparatus1 is connected with two peripherals 2A and 2B. One 2A of the twoperipherals is used for operating the setting while the other 2Brequests downloading of the measurement data. Displayed on the screen ofthe image processing apparatus 1 are the image 7 of the object to beprocessed, the menu 8, and a text data 9 indicative of the status of itsown and each peripheral 2. The display screens 200A and 200B of theperipherals 2A and 2B provide two windows 26A and 26B respectively wherethe display image is displayed. The screen 200B of the peripheral 2Balso provides another window 28 for displaying the downloading actionbeing conducted. The window 28 is selectively displayed by theperipheral 2B while it is absent on the other screens 100 and 200A.Similarly, the simulated commander 27A is selectively turned on by theperipheral A which commands the setting action, thus permitting theinverted display of relevant buttons or a cursor as denoted by 29 in thefigure.

FIG. 20 illustrates a combination of actions between the peripheral 2and the image processing apparatus 1. This combination includes asequence of steps conducted between the single peripheral 2 and theimage processing apparatus 1, provided that the communicationcontrolling action shown in FIG. 7 is executed.

FIG. 20 illustrates the action of the image processing apparatus 1divided into three functions of controlling the communication with theperipheral 2 (at a communication controller), controlling the display onthe monitor (at a display controller), and controlling the measurementaction (at a measurement controller). The three controllers are operatedin parallel. The steps (ST) of the action of the communicationcontroller are denoted after 100, of the display controller are denotedafter 200, and of the measurement controller are denoted after 300. Theaction of the peripheral 2 includes Steps (ST) 1 to 11. The relationshipbetween two particular steps is expressed by the solid arrow in thefigure. For ease of the description about the setting acceptancecommand, the action of the image processing apparatus 1 will beexplained with a procedure of updating the display image.

Starting at ST 301, the measurement controller in the image processingapparatus 1 repeats three consecutive steps of acquiring the image fromthe camera 11 (ST302), saving the image in the image memory 112 (ST303),and carrying out the measurement action over the image (ST304).

Starting at ST201, the display controller receives the image acquired ora modification of the image generated by the measurement action (ST202).It is then examined at ST203 whether the setting action is made or not.When the console 12 is operated or the setting acceptance command isdispatched from the peripheral 2, it is judged that the setting actionis made. When it is judged “yes” at ST203, the procedure advances toST204 where an image to be displayed is generated from the imageacquired at ST202 and accompanied with the menu of setting or otherinformation. When its is judged at ST203 that the setting action is notmade, the procedure goes to ST205 where the image acquired at ST202 isdesignated as the display image.

This is followed by ST206 for saving in the image memory 112 the displayimage determined at ST204 or ST205. Also, the display image is displayedon the monitor screen at ST207.

The communication controller in the image processing apparatus 1 startsat ST101 and waits for receiving a connection command from theperipheral 2 (ST101-102). The connection command is a set of dataincluding a registered password. When the connection command is receivedfrom the peripheral 2, it is judged “yes” at ST103. Then, the acceptanceof the connection is transmitted the peripheral 2 at ST104.

This is followed by ST105 for waiting for receiving the imagetransmission command from the peripheral 2. When the image transmissioncommand is received, the procedure moves from ST106 to ST107 where thedisplay image saved at ST206 is read out. Meanwhile, as thecommunication controller, the display controller, and the measurementcontroller are operated in parallel, the display image obtained at ST107is the latest image generated by the display controller. Also, thelatest display image is generated from the latest image acquired by themeasurement controller.

ST108 follows where the display image read out is subjected to thecompression and encryption process. Then, the response is generatedcarrying the compressed and encrypted display image in the body andtransmitted to the peripheral 2 at ST109.

The action of the peripheral 2 will be described. The action of theperipheral 2 starts with turning the power on, dispatching theconnection command, and confirming the connected accepted by theresponse from the communication controller (ST1 to ST3). Then, the loopST4 to ST8 and the loop ST9 to ST11 are executed in parallel.

The loop ST4 to ST8 is designed for controlling the display of theimage. At ST4, the image transmission command is dispatched to the imageprocessing apparatus 1. In turn, the communication controller in theimage processing apparatus 1 performs the action of ST106 to ST109 fortransmitting the response. Then, as the response is received as asequence of packets, the receipt completion message is dispatched atST5. After receipt of the response, the action moves to ST6 fordecompressing and decoding the response to reconstruct the displayimage. This is followed by ST8 where the reconstructed display image isdisplayed on the monitor screen.

The above steps are repeated until the completion of the display isdesired. Accordingly, the latest display image can be received from theimage processing apparatus 1 and displayed by operating the former loop.

Although the decompression and decoding of the image data is conductedafter the action of receipt at ST5 in the above loop, it may be executedupon receipt of each packet.

In the latter loop ST9 to ST11, the action of examination at ST9 isrepeated until the setting action with the simulated commander 27starts. When the setting action starts, the loop actionmoves from ST9 toST10 where the setting acceptance command is dispatched to the imageprocessing apparatus 1. The setting acceptance command is actuallyreceived by the display controller in the image processing apparatus 1.The display controller then carries out the action of ST203 and ST204.At ST206, the display image including the data of the setting action issaved. The display image including the data of the setting action isread out and dispatched to the peripheral 2 by the communicationcontroller conducting the action of ST107 in response to the imagetransmission command which follows the setting acceptance command.

For executing the communication controlling action shown in FIGS. 8 to10, the image processing apparatus 1 and the peripherals 2 carry outsubstantially the steps shown in FIG. 20 for exchanging the data.Therefore, discrepancies only will be described in brief.

For the communication controlling action shown in FIG. 8, thecommunication controller in the image processing apparatus 1 waits atST109 for a given length of time before transmitting the response. Whenthe image transmission command is received from another peripheral 2during the waiting period, the response to the image transmissioncommand is transmitted to each of the peripherals 2 after the waitingperiod. The action of the peripherals 2 remains unchanged.

For the controlling action shown in FIG. 9, the peripheral 2 performsthe dispatching of the first image transmission command and measures theduration for processing before displaying the image received in responseto the command. The, the second image transmission command is dispatchedtogether with the measurement of the processing duration. Then, theperipheral 2 waits for receiving the direct response from the imageprocessing apparatus 1 and carried out the action of displaying thedisplay image upon receiving the response. At the time, thecommunication controller in the image processing apparatus 1 executesthe action of ST107 to ST109 in response to the image transmissioncommand excluding the measurement of the processing duration. When theimage transmission command including the measurement of the processingduration, the action of ST107 to ST109 is carried out whenever theprocessing duration is elapsed.

For the controlling action shown in FIG. 10, the peripheral 2 confirmsthe connection at ST3 and then conducts the action of ST4. Then, theaction of ST4 is repeated until the response from the image processingapparatus 1 is received. When the action of receiving the response atST5 is completed, the action of ST6 and ST7 is preceded by dispatchingthe image transmission command. In this case, the action of the imageprocessing apparatus 1 remains unchanged.

For the communication controlling action shown in FIG. 11, the steps aremodified as shown in FIG. 21. The modification of the steps is madebetween the communication controller and the peripheral 2 while theother steps are identical to those shown in FIG. 20. Hence, like stepsin FIG. 21 are denoted by like numerals as those shown in FIG. 20 andwill be explained for discrepancies.

The communication controller in the image processing apparatus 1 whenreceiving the image transmission command at ST106 repeats the action ofST107 to ST109. The action of ST107 and ST108 is equivalent to that ofStep A shown in FIG. 12. The action of ST109 is equivalent to that ofSteps B to E shown in FIG. 12.

After dispatching the image transmission command at ST4, the peripheral2 receives the response to the command and performs the action of ST5 toST8. When the display action is completed, the action returns from ST8to ST5 for repeating the receipt action, the decompressing and decodingprocess, and the display action upon receiving the response to thesecond command from the communication controller. Similarly, the actionof ST5 to ST8 is repeated until “yes” stands at ST8.

The steps shown in FIG. 21 are applicable to the communicationcontrolling action shown in FIG. 13. In the action, the communicationcontroller in the image processing apparatus 1 when receiving the imagetransmission command from a particular peripheral 2 starts the action ofST107 to ST109 for the peripheral 2 which dispatches the command.Otherwise, the action of ST107 to ST109 is conducted at once for all theperipherals 2 being connected. The action of the peripherals 2 remainsunchanged.

1. A method of controlling an image processing system which consistsmainly of an image processing apparatus and a desired number ofperipheral apparatuses connected together over a computer network forcarrying out a given measurement action, wherein the image processingapparatus repeats a first step of generating an image to be processed, asecond step of generating a display image from the image to beprocessed, and a third step of saving the display image in a memory, andselectively performs, provided that an image transmission command basedon a stateless protocol is received from any peripheral apparatus, afourth step of extracting a desired number of the display images fromthe latest saved in the memory and a fifth step of generating andtransmitting to the peripheral apparatus a response of a specific formatto the image transmission command, predetermined by the image processingapparatus, which contains the extracted display images but not a controldata of the peripheral, and the peripheral apparatus performs a step Aof dispatching to the image processing apparatus the image transmissioncommand of a specific format predetermined by the peripheral apparatus,a step B of receiving the response to the image transmission commandfrom the image processing apparatus, and a step C of displaying thedisplay image, carried in the response, at a specific formatpredetermined by the peripheral apparatus on its monitor.
 2. A method ofcontrolling an image processing system, according to claim 1, whereinthe image processing apparatus performs the fourth and fifth stepswhenever receiving the image transmission command from the peripheralapparatus, and the peripheral apparatus repeats the step A and the stepB in this order and performs the step C while repeating the step B.
 3. Amethod of controlling an image processing system, according to claim 1,wherein the image processing apparatus performs the fourth and fifthsteps whenever receiving the image transmission command from theperipheral apparatus, and the peripheral apparatus repeats the step Aand the step B in order of B and A after executing the step A at leastone time and, as preceded by the step A, performs the step C whilerepeating the step B.
 4. A method of controlling an image processingsystem, according to claim 1, wherein the image processing apparatuswaits for a given period of time when receiving the image transmissioncommand from one peripheral apparatus and, when receiving the imagetransmission command from another peripheral apparatus during thewaiting period, transmits the common response to both the one andanother peripheral apparatuses.
 5. A method of controlling an imageprocessing system, according to claim 1, wherein the peripheralapparatus performs one cycle of the steps A, B, and C, determines fromthe execution of the cycle a processing period of time needed forcommunicating with the image processing apparatus, and performs a stepDof dispatching a secondary image transmission command including theprocessing period, the image processing apparatus performs the fourthand fifth steps one time in response to the image transmission commandincluding non of the processing period and when receiving the secondaryimage transmission command including the processing period, performs thefourth and fifth steps whenever the processing period is elapsed, andthe peripheral apparatus repeats the steps B and C after completing thestep D.
 6. A method of controlling an image processing system, accordingto claim 5, wherein the peripheral apparatus measures a period of timein which one cycle of the steps B and C is executed during performingthe steps B and C after the step D and when the measured period isshorter by a given length than the processing period, repeats the step Dthrough updating the processing period with the measured period, and theimage processing apparatus when receiving the secondary imagetransmission command including an updated version of the processingperiod, modifies the interval between the fourth step and the fifth stepaccording to the updated version of the processing period.
 7. A methodof controlling an image processing system, according to claim 1, whereinthe image processing apparatus repeats the fourth and fifth steps inresponse to the image transmission command from the peripheralapparatus, and the peripheral apparatus performs the step A one time andperforms the steps B and C whenever receiving the response from theimage processing apparatus.
 8. A method of controlling an imageprocessing system, according to claim 7, wherein the peripheralapparatus, when the receipt of the response is completed at the step B,performs an action of notifying the image processing apparatus of thecompletion of the receipt, and the image processing apparatusacknowledges the end of the fifth step from the notification of thereceipt completion from the peripheral apparatus and repeats the fourthstep.
 9. A method of controlling an image processing system, accordingto claim 1, wherein the peripheral apparatus, when the setting actionfor the image processing apparatus is conducted at its operation panel,performs a step E of generating and dispatching an setting acceptancecommand including the setting data to the image processing apparatus,and the image processing apparatus when receiving the setting acceptancecommand, performs a sixth step of updating the display image with thesetting data carried in the command and a seventh step of saving theupdated display image in the memory.
 10. A method of controlling animage processing system, according to claim 9, wherein the settingaction includes selecting items for the setting, and the imageprocessing apparatus generates at the sixth step the display imageincluding a menu of the items for the setting when receiving the settingacceptance command including the setting data.
 11. An image processingsystem comprising an image processing apparatus and a desired number ofperipheral apparatuses connected together over a computer network forcarrying out a given measurement action, wherein the image processingapparatus includes an image inputting device for inputting an image tobe processed, a display image generating device for generating a displayimage from the image input by the image inputting device, a memory forsaving the display image, a command receiving device for receiving animage transmission command based on a stateless protocol from anyperipheral apparatus, an image extracting device for extracting adesired number of the display images from the latest saved in thememory, a responding device for generating and transmitting to theperipheral apparatus a response of a specific format to the imagetransmission command, predetermined by the image processing apparatus,which contains the extracted display images but not a control data ofthe peripheral apparatus, and a controlling device for actuating theimage extracting device and the responding device, provided that thecommand receiving device receives the image transmission command, andthe peripheral apparatus includes a command dispatching device fordispatching to the image processing apparatus the image transmissioncommand of a specific format predetermined by the peripheral apparatus,a response receiving device for receiving the response to the imagetransmission command from the image processing apparatus, and a displaycontrolling device for displaying the display image, carried in theresponse, at a specific format predetermined by the peripheralapparatus.
 12. An image processing system according to claim 11, whereinthe controlling device in the image processing apparatus actuates theimage extracting device and the responding device whenever the commandreceiving device receives the image transmission command from theperipheral apparatus, and the command dispatching device in theperipheral apparatus dispatches another image transmission commandwhenever the receiving device receives the response and the displaycontrolling device displays the display image.
 13. An image processingsystem according to claim 11, wherein the controlling device in theimage processing apparatus actuates the image extracting device and theresponding device whenever the command receiving device receives theimage transmission command from the peripheral apparatus, and thecommand dispatching device in the peripheral apparatus conducts thedispatching of the image transmission command at least one time anddispatches another image transmission command prior to the displaycontrolling device displaying the display image, whenever the receivingdevice receives the response.
 14. An image processing system accordingto claim 11, wherein the controlling device in the image processingapparatus directs the responding device to wait for a given period oftime when the command receiving device receives the image transmissioncommand from one peripheral apparatus and, when receiving the imagetransmission command from another peripheral apparatus during thewaiting period, transmits the common response to both the one andanother peripheral apparatuses.
 15. An image processing system accordingto claim 11, wherein the peripheral apparatus includes a transmissioncommand controlling device for performing one cycle of the actions ofthe command dispatching device, the response receiving device, and thedisplay controlling device, determining from the execution of the cyclea processing period of time needed for communicating with the imageprocessing apparatus, and directing the command dispatching device todispatch a secondary image transmission command including the processingperiod, and the controlling device in the image processing apparatusdirects the image extracting device and the responding device to performtheir action one time in response to the image transmission commandincluding non of the processing period and when receiving the secondaryimage transmission command including the processing period, directs theimage extracting device and the responding device to perform theiraction whenever the processing period is elapsed.
 16. An imageprocessing system according to claim 15, wherein the transmissioncommand controlling device in the peripheral apparatus includes ameasuring device for measuring a period of time in which one cycle ofthe actions of the response receiving device and the display controllingdevice is executed when the secondary image transmission commandincluding the processing period has been received, and a processingperiod updating device for updating the processing period with themeasured period when the measured period is shorter by a given lengththan the processing period and then directs the command dispatchingdevice to dispatch a further image transmission command including theupdated processing period, the controlling device in the imageprocessing apparatus when receiving the further image transmissioncommand including the updated processing period, modifies the intervalbetween the action of the image extracting device and the action of theresponding device according to the updated processing period.
 17. Animage processing system according to claim 11, wherein the controllingdevice in the image processing apparatus directs the image extractingdevice and the responding device to repeat their action in response tothe image transmission command received by the command receiving device,and the peripheral apparatus includes a dispatch controlling device fordirecting the command dispatching device to dispatch the imagetransmission command one time and then actuating the response receivingdevice and the display controlling device whenever the response isreceived from the image processing apparatus.
 18. An image processingsystem according to claim 17, wherein the dispatch controlling device inthe peripheral apparatus, when the receipt of the response by theresponse receiving device is completed, performs an action of notifyingthe image processing apparatus of the completion of the receipt, and thecontrolling device in the image processing apparatus includes a devicefor receiving the notification of the receipt completion from theperipheral apparatus and acknowledges the end of the action of theresponding device from the notification of the receipt completion andthen restarts the action of the image extracting device.
 19. An imageprocessing system according to claim 11, wherein the peripheralapparatus includes an inputting device for inputting the setting datafor the image processing device and an setting acceptance commanddispatching device for, when the setting data for the image processingapparatus is input by the setting action of the inputting device,generating and dispatching an setting acceptance command including thesetting data to the image processing apparatus, and the display imagegenerating device in the image processing apparatus, when the settingacceptance command is received by the receiving device, updates thedisplay image with the setting data carried in the command and saves theupdated display image in the memory.
 20. An image processing systemaccording to claim 19, wherein the setting action includes selectingitems for the setting, and the display image generating device in theimage processing apparatus generates the display image including a menuof the items for the setting when receiving the setting acceptancecommand including the setting data.
 21. A program for functioning acomputer as a peripheral apparatus connected over a computer network toan image processing apparatus for performing a measurement action,whereby the computer is assigned with a command dispatching device fordispatching to the image processing apparatus an image transmissioncommand at a specific format or stateless protocol format predeterminedby the peripheral apparatus, a response receiving device for receiving aresponse to the image transmission command from the image processingapparatus, and a display controlling device for displaying a displayimage, carried in the response, at a specific format predetermined bythe peripheral apparatus on its monitor screen.